In targeting, such as behavioral targeting, historical information such as online user behavior information can be used in targeting high-performing advertisements to users. In this regard, a taxonomy, such as a hierarchical taxonomical tree of categories or topics, may be utilized, in which nodes in the taxonomy may represent categories of interest or behavioral targeting categories. A machine learning-based model may be utilized in selecting advertisements for matching with serving opportunities and for serving to particular users. Historical user behavior information may be used to train the model offline. For example, such models may be trained periodically, such as monthly, weekly, or even more frequently, using updated historical user behavior information.
Online, the offline-trained model may be utilized in advertisement targeting and in determining or facilitating determination of such things as serving thresholds and delivery policies. Serving thresholds may include scores, which may directly correspond to particular CTRs, for instance. The serving thresholds may, for instance, relate to certain minimum scores or CTRs that will be required for serving of an advertisement in a particular category of the taxonomy, for instance (of course, there are many details in the process which are not described here, for simplicity of explanation). Delivery, policies may, for instance, govern serving based at least in part on available advertising inventory, or available inventory for a particular taxonomical category, etc. For example, serving thresholds may be set in each category in such a way that both a desired level of performance, measured by category-specific CTR, and a desired volume of deliverable ad impressions in the same category, are predicted to be achieved. Naturally, for a large-scale operation, scores and thresholds may be set based on many other factors as well, and will take into account many other variables across many advertisers, campaigns, etc. Generally, the model may be used in making predictions and projections based on the offline training.
Generally, serving thresholds and delivery policies are determined offline. Online, real-time or near real-time information, including newly obtained user behavior information, etc., can be fed into the model, and the model can be used in determining when circumstances are right for serving, such as when a particular serving opportunity to a particular user is predicted or projected by the model to meet requirements such as the predetermined serving thresholds and delivery policy requirements.
As mentioned, such models are generally refreshed periodically by offline training with newly collected historical user behavior information. Models may only be refreshed so frequently, such as monthly, weekly, or perhaps daily. However, circumstances, events, and developments occur and change dynamically in real time, and the model cannot be refreshed constantly to include such information as training information. Such dynamic developments may include, as just one example, a breaking news event, which may affect anticipated CTRs or pertain to optimal delivery policies, etc. Existing methods utilizing, for example, serving thresholds and delivery policies set using models that may have been refreshed offline may lead to suboptimal serving-related decision-making and determinations.
There is a need for methods and systems for improving or optimizing serving decision-making and determinations and associated serving thresholds and delivery policies.